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A RESEARCH PROJECT USING THE OSCILLOSCOPE TO DETERMINE 
VISUAL FEEDBACK IN THE TEACHING OF FOREIGN LANGUAGE 
PRONUNCIATION WAS TERMINATED BECAUSE OF TECHNICAL 
DIFFICULTIES THAT COULD NOT BE RESOLVED WITH THE EQUIPMENT 



AVAILABLE. 
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TO SUCH FACTORS AS (1) THE 



SPEECH SOUND WAVES SOUND THE SAME THOUGH THEIR WAVE SHAPES 



DIFFER, (2) THE OSCILLOSCOPE DOES NOT PRESENT AN EASILY 



PERCEIVED PITCH AS DOES AURAL FEEDBACK, (3) THE SUBJECT MUST 
AWAIT THE CRYSTALLIZATION OF THE VISUAL IMAGE WHICH IS 
COMPARABLE TO THE PROBLEM OF STAMMERING IN AURAL FEEDBACK, 
AND (4) THE LOSS OF VISUAL RECOGNITION APPEARS TO BE GREATER 



THAN LOSS OF AUDITORY RECOGNITION. BEFORE AN ATTEMPT CAN BE 



MADE TO DETERMINE THE USEFULNESS OF ANY INSTRUMENTATION FOR 
FOREIGN LANGUAGE PRONUNCIATION INSTRUCTION, IT IS RECOMMENDED 
THAT FUTURE RESEARCH AIM FOR VISUAL SPEECH RECOGNITION RATHER 
THAN EXACT VISUAL REPRODUCTION THROUGH FURTHER ANALYSIS OF 
ALL THE COMPONENTS OF PHONEME STRUCTURE, VARIABILITY, AND 
CONSISTENCY. APPENDIXES LIST THE EQUIPMENT USED IN THE 



EXPERIMENT, THE EXPENDITURES, AND A BIBLIOGRAPHY. (SS) 
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I* General 

Research on Project No* 2685 has been terminated since results 
vith an experimental group on the effect of visual feedback on 
pronunciation learning have been invalidated by technical difficulties 
beyond the resolving powers of this researcher at this time* 

The design of the project w a predicated on research published by 
Barton and Barton It involved the visual presentation of phonemes 
in English by meanB of oscilloscopic images developed through a 
relatively simple resistor-capacitor input- network* The use of the 
oscilloscope Itself is not novel for this purpose* The work of 
Barton and Barton was, however, unusual in the claim for the 
distinctiveness claimed for the visual image of phonemes and the 
simplicity of the apparatus utilized* 

It was contemplated by Dr. Paul Jenson and myself that tho project 

as proposed by us would utilize, therefore, basic equipment and 

instrumentation already developed by others. Our attempt would 

» 

concei '-.rate on applying this technique and equipment in the teaching 
of pronunciation lor foreign languages. 



1* Barton and Barton, 'Forms of Sounds as Shown on an Oscilloscope 
by Hewlett© Figures, 1 Science , Yol. 142, pp. 1455-1456, 

f 



2 



»* 



A test of pronunciation in German was developed as part of the 
project, based on work published by William G* Moulton® in 
connection with research performed for the United States Office of 
Education and principles as proposed by Robert Lado*3 The ultimate 
criterion was communication effectiveness* Test items were chosen 
from basic vocabularies for beginning German courses and resulted in 
the selection of 64 single syllable and spondee words* Phonemic 

i 

contrast and phonetic balance were strong factors in choice* 
Inter-rater agreement reached 90$ on one form of the test and of the 
64 items only two items had to be discarded* 

n. Instrumentation Problems 

The crux of the problem from the beginning to this time has been 
the failure of the instrumentation technique employed by Barton and 
Barton to be effective as has been noted by others working in the 
field. Barton and Barton were well aware of the necessity for 
proceeding beyond mere frequency versus time plotting* In fact, their 

technique degraded pitch presentation as well as losing time duration 

» 

of sound components in favor of showing phase relationships* A 

/ 



2* Moulton, William G«, The Sounds of English and Germ an « 
University of Chicago Press, 1952 

5* Late, Robert , * Language Testing . McGraw-Hill Inc*, 1964 



» 



3 



practical test of their system when applied to the phonemes of 
General American English failed to he reasonably comparable between 
different sample speakers as well as falling to produce invariance 
of visual imago on the same phoneme. 

There is little doubt that the oscilloscope is an instrument without 
peer in the study of any phenomena -which can be converted to electrical 
impulses and then examined in regard to wave form; that is, frequency, 
amplitude, and phase relationships# However, as experimentation with 
instrumentation continued. It also became clearer that the problems 
were much more complex than either researcher had envisioned# 

Digital display by means of a frequency counter was attempted but 
proved ineffective for the some reasons as to be given in regard to 
oscllloscoplc images* 

The basic fallacy of oocilloscopic utilization lies in itG presenta- 
tion of wave shapes and not the speech spectrum# Denes and Pinson 
of the Bell Telephone Laboratories have pointed out the advantage of 
spectographlc presentation; a method which also has strong limitations 

In using the oscilloscope for visual feedback in the teaching of 
pronunciation, failure appears to be attributable to the following t 



4# Denes, Peter B* and Pinson, Elliot N«, The Speech Chain# Bell 
Telephone Laboratories, 19 63 
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1* Speech Bound images of sinusoidal waves often sound the 
same even though their vuve shapes differ because of 
differences in the phase relationship of their components# 

2# Aural pitch perception is not proportional to frequency 
but the oscilloscope visual presentation is proportional 
and, therefore, does not present an easily perceived scale 
when combined with aural feedback# 

3# Aural perception of amplitvi&e changes suffers frcm a similar 
problem with the additional handicap that the large varia- 
tions in amplitude between different phonemes demand scale 
changes in the oscilloscope amplification factor# 

4# The speech feed-back affect seems to have its parallel in 
visual feed-back# That is, the subject awaits the crystalli- 
zation of the visual image thus leading to a concomitant of 
the stammering effect of delayed aural feed-back# 
Characteristic patterns were developed for phonemes which could be 

duplicated by the speaker again and again but the communicative test 

* \ 

proved that the visual Images lost their identity much sooner than the 

auditory image. Further the pattern changes from subject to subject 

/ 

caused a loss of visual image recognition when auditory recognition 

was still within the virtual certainty range# By limiting the audio 

band pass width to a range of 1000 Hertz , as suggested by Denes and 

Pinson, their findings were confirmed in regard to aural intelligibility 

but corresponding visual images were attenuated beyond consistent 

» 



recognition oven for a single subject. 



HI# Formant Theory 

The difficulties encountered with the oscilloscope, as above, led to 
a concomitant examination of the formant theory of speech sound 
classification. It is commonly claimed that the first, second, and 
third formants arc visually sufficient for recognition of vowel phonemes. 
Unfortunately, this docs not mean that a particular combination of 
formant frequencies will always he recognized as one and the same 
vowel. Further a vowel can he aurally recognized even though its 
formant frequencies vary widely. As Denes and Pinson have pointed 
out and, as verified here, formant frequencies do not positively 
identify a vowel. In fact, no acoustic feature can completely 
identify a speech sound. William E. Castle published a study on the 
possible invariance of formant structure denying the absolute theory 

as first suggested by HeMholtz. 5 in reference to particular vowel 

BO unas he found that formant information per se proves less useful to 

* 

listeners than does information offered to their ears through 
third-octave bandwidth which center at 1600 or 2000 Hertz. This 
finding was confirmed in visual presentation by use of a Kronhite 
variable band-pass filter. These problems are further accentuated 



tt m castle. W illi am E., 1 A Cement on the Possible { invariance of 
5 Formant Structure for Specific Vowel Phonemes,* linguistics, 
Volt 13, April 1965, PP* 1^-24 
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when there is doubt concerning the formant spectrum itself as evidenced , 
by the work of Delattre in revising formant definition for French 
phonemes and the phonetic comparison of languages *6 

Through application of presently available band-pass filters, a 
number of researchers have quite accurately determined the frequency 
points at which communication intelligence ceases and what frequencies 
may he eliminated without serious loss* However, what has been 
deleted, or remains, is not only a component of a particular frequency 
or band of frequencies but an entire complex speech block of which the 
prosodic features are a significant, component* A de-emphasis of 
formant components is reflected in the work of Buiten and lane on a 
self-instructional device for conditioning accurate prosody *7 Intonation, 
stress and rythmn values are compared by the subject to a model by 
means of simple digital presentation developed by a highly complex 
system of instrumentation* Research referenced by A. P. van Teslaar 
in a previous issue of IRAL deal3 with manipulating the frequency 
curve of the feedback signal to give prominence to certain acoustic 

a ’ 

features • It will be interesting to note If such research will also 

/ 



6. Delattre, Pierre, 'Research Techniques for Phonetic Comparison of 
Languages,' IRAL, Vol* 1/2, 1963, pp* 85-97 

7* Buiten, Roger and Lane, Harlan, *A self-instructional Device for 

Conditioning Accurate Prosody, » IRAL , Vol. III/3 (1965), pp* 205-219 

* j 

8 * Tan leul^ar, A* P., 'Learning Sew Sound Systems i Problems and 
Prospects,' IRAL. Vol. XH/2, 1 965, pp. 79-93 
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produce information on the significant parameters of phoneme identity* 

Reliance on "Formant" content alone is fallacious with present formant ' 
definition since t 

1. A wide range of formant frequencies is recognized as the 

same phoneme • / 

2* Formant ranges for phonemes overlap. 

3* Formants for any one speech sound vary from speaker to 
speaker* 

4# Formants are significantly influenced by preceding and 
following sounds. 

5. A particular phoneme is not invariably associated with a ^ 

particular combination of formant frequencies * 

IV* Practical Application 

It was hoped that this research would produce a relatively inexpensive 
and simple practical aid for foreign language instruction; instrumen- 
tation that would be included in the language laboratory alongside 
the tape recorder or the auto-record machinery of Chester Electronic : 

Laboratories* If a solution is found in the instrumentation area, 
it is doubtful whether it will be either simple or inexpensive* 

Spectographic instruments available at this time are neither* 

OsciUoscopic presentation or digital read-out still appear as possible 
avenues of attack if the intermediate instrumentation can be resolved* 

For the purpose of practical application any future research should 
not have as Its aim exact visual reproduction but visual speech 
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recognition. In the meantime, I believe, that further expenditure of 
funds on applied research is premature and that funds should, ho 
directed into basic research leading to practical instrumentation. 

4 

Such basic research should probably be first directed at instrumental 
isolation of all the components of phoneme structure, their clear 
definition ana consequently at a test of variability and consistency. 
In any case, the engineer-technician must be given a clear presenta- 
tion of what Ms Instrument will define visually. 

V. Future Activities 

Since the contract did not permit capital purchases, instruments 
were acquired both by Macalester College and myself in order to 
carry on this research. I hope to continue the use of procured 
equipment and space allotted to me in continuation of research in this 
area. Two instrumentation techniques are planned for investigation. 

First, the utilization of Mgh-speed recording; that is, taps speeds 

of 60 to 120 inches . This will permit easier physical manipulation 

of recorded sample#. Secondly, analog to digital conversion. Up to 

the present time, research has been accomplished almost entirely with 

analog read-out, with ie exception, os already noted, of the work of 

Ur. Harlan Lane. Through the use of pulse code modulation, as 

proposed by Reeves of ITT, the continuously variable speech signals 

may be transformed into a series of digitally coded pulses . This 

transformation implies increased band width in the transmission of 

♦ 
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speech and may allow Improved Isolation and definition techniques. 

It is further hoped, as the research of Professor Lane with "SAID 1 * 
continues into use in foreign language instruction, that the place 
of the supras egmont al features of speech he more clearly def ine d » 
The parameters of segmental* may thus he limited by extraction and 
more easily attacked. 
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Appendix A 



Experimentation liras carried out with the use of .the following major 
equipment: 

Oscilloscopes 



564 Storage Oscilloscope* including type 
2 VT 2 dual trace amplifier and type 2*367 time hose* 

"Dumont” Type 401 * 



Pulse Generator 

"Electro-Pulse Inc*" . Model 2125B 
Signal Generator (Audio) 

"Continental Electronics" Type SG-15A/PCM 
"Hewlett Packard" 

Decibel Meter 



"Byron Jackson" Type ME-22/V/pCM 
Voltage Frequency Converter 
"Dymec Inc*" Model Dy-2210 
Variable Band-Pass Filters 
• "Kronldte" 

/ 

"United Transformer Corporation" 
Stereo Tape Recorder 

"Chester Electronic Laboratories" 
Vacuum Tube Voltmeter 

"Ballantine Laboratories" Model 64j 

Electronic Switch 

"Heath Company" Model S -2 
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5,160.00 

85,00 
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Supplies and Materials 294 *jj3 
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